X-ray inspection system that integrates manifest data with imaging/detection processing

ABSTRACT

The present specification discloses systems and methods for integrating manifest data for cargo and light vehicles with their X-ray images generated during scanning Manifest data is automatically imported into the system for each shipment, and helps the security personnel to quickly determine the contents of cargo. In case of a mismatch between cargo contents shown by manifest data and the X-ray images, the cargo may be withheld for further inspection. In one embodiment, the process of analyzing the X-ray image of the cargo in conjunction with the manifest data is automated.

CROSS-REFERENCE

The present specification relies on U.S. patent provisional applicationNo. 61/532,093, which, filed on Sep. 7, 2011, and entitled “X-RayInspection System with Integration of Manifest Data withImaging/Detection Algorithms”. The above referenced application isincorporated herein by reference in its entirety.

FIELD

The present specification discloses systems for inspecting goods incontainers and, more specifically, to systems that integrate cargomanifest data with imaging and/or detection processes to make inspectiondecisions and/or generate alarms upon detecting the presence of threatitems in cargo.

BACKGROUND

Cargo containers need to be inspected at ports and other points of entryor transportation for contraband such as explosives, narcotics,currency, chemical and nuclear weapons and for cargo-manifestverification. A cargo manifest is a physical or electronic shippingdocument that accompanies the cargo and provides important descriptiveinformation about the cargo, including bills of lading issued by thecarrier or its representative(s), the shipment's cosigner and/orconsignee, cargo description, amount, value, origin, and/or destination.The accurate detection of contraband with a low false alarm is adaunting task, as these materials often have similar physicalcharacteristics as benign cargo. The percentage of cargo to be inspectedis increasing, and because of the currently manually intensive nature ofinspections, so is the number of operators.

Security systems are thus limited in their ability to detect contraband,weapons, explosives, and other dangerous objects concealed in cargo.Standard and advanced X-ray systems have difficulty detecting contrabandin break-bulk cargo. This difficulty is exacerbated when inspectinglarger and oftentimes, cluttered pallets and cargo containers. ComputedTomography (CT) based systems have been shown to be more suitable forthe difficult task of detecting aviation-threat explosives in luggageand, more recently, in larger objects. However, the configuration ofcommonly employed CT systems prevents scaling the system up to longobjects such as large cargo containers and large skids.

The problem is further compounded by the fact that as a result of theimage modulation according to atomic numbers of various materials, it iscommon for X-ray imaging systems to produce images with dark areas.Although these dark areas might indicate the presence of threatmaterials, they yield little information about the exact nature ofthreat. Also, radiographs produced by conventional X-ray systems areoften difficult to interpret because objects are superimposed.Therefore, a trained operator must study and interpret each image torender an opinion on whether or not a target of interest, a threat, ispresent. Operator fatigue and distraction can compromise detectionperformance, especially when a large number of such radiographs is to beinterpreted, such as at high traffic transit points and ports. Even withautomated systems, it becomes difficult to comply with the impliedrequirement to keep the number of false alarms low, when the system isoperated at high throughputs.

Therefore, there is a need to provide an automated detection system thatfurther includes assistance tools to help operators improve theirthroughput by scrutinizing cargo images more efficiently, therebyincreasing detection and analysis speed. There is also a need for suchsystems to operate with reduced false alarm rates.

SUMMARY

The present application discloses a system for associating andintegrating manifest data from cargo and light vehicles with theirrespective X-ray images that are generated during scanning. Manifestdata is automatically imported into the system for each shipment, andhelps the security personnel to quickly determine the contents of cargo.In case of a mismatch between cargo contents shown by manifest data andthe X-ray images, the cargo may be withheld for further inspection.

In one embodiment, manifest data is imported via an applicationintegrated within an X-ray detection system deployed at checkpoints orservice posts. In one embodiment, the application works within theframework of a distributed network, wherein the service post isconnected to a regional center, where an operator can analyze the X-rayimage of the cargo in conjunction with the manifest data. When the X-rayimage and manifest data has been analyzed, the service post whichperformed the non-intrusive X-ray scan will be notified automatically bythe application integrated with the X-ray system. This allows theservice post operator to make a decision to either release the cargo orto hold the cargo for further inspection.

In one embodiment, the process of analyzing the X-ray image of the cargoin conjunction with the manifest data is automated.

In one embodiment, the present specification discloses a system forscanning cargo and vehicles, comprising: at least one non-intrusiveinspection system for performing a non-intrusive X-ray scan, saidnon-intrusive inspection system further comprising an application forimporting manifest data associated with the cargo or vehicle beingscanned; and a processing system for receiving scan images andassociated manifest data from the non-intrusive inspection system, anddetermining from the scan images if the contents of the cargo or vehicleare of the same type as specified in the manifest data. In oneembodiment, the system further comprises a server that executes anapplication for allocating images and manifest data from a service postto a regional center. Further, in one embodiment, each X-ray scan imageis associated with a unique identifier before transmission from theservice post to the regional center.

In one embodiment, the scan images and manifest data are analyzed by anoperator at a regional center to determine if the contents match.Further, the scan images and manifest data are automatically analyzed byan application at a regional center to determine if the contents match.

In one embodiment, scan images, their associated unique identifiers andmanifest data, and results of analyses at a regional center are storedin a database.

In another embodiment, the present specification discloses a method forinspecting cargo and vehicles, comprising: scanning a cargo container orvehicle at a service post using non-intrusive system; importing manifestdata associated with the cargo or vehicle being scanned; and analyzingsaid scan images and associated manifest data to determine if thecontents of the cargo or vehicle correspond to the same cargo type asspecified in the manifest data. In one embodiment, the non-intrusivescanning is performed by an X-ray system.

In another embodiment, the step of analyzing further comprisesdetermining whether a threat item or alarm condition is present.Further, the result of the analysis at a regional center is reported tothe service post. Still further, each scan image is associated with aunique identifier before transmission from the service post to aregional center. In addition, the scan images, their associated uniqueidentifiers and manifest data, and results of analyses are stored in adatabase.

In yet another embodiment, the present specification discloses a methodfor screening cargo, the method comprising: scanning a cargo containerusing non-intrusive X-ray system to generate a scan image; importingmanifest data associated with the cargo or vehicle being scanned;obtaining cargo code information from manifest data; retrieving storedimages associated with said cargo code from an image database; andcomparing features of said scan image to features of historically storedimages to determine if the contents of the cargo or vehicle match themanifest data. In addition, the method further comprises computingfeatures of cargo content from the generated scan image, including butnot limited to, attenuation, texture, atomic number, cargo height,density and atomic number. In one embodiment, the step of comparingfurther comprises comparing the computed features from the scan imagewith features associated with historically stored images.

In one embodiment, the computed features are associated with said cargocode, wherein the cargo code is indicative of type of cargo. Further,the method comprises segregating the generated scan image according tocargo types, if the cargo is associated with more than one cargo code.Still further, each segregated part of the image is compared tohistorically stored images associated with the corresponding cargo code.Yet still further, the step of comparing said scan image to historicallystored images is performed automatically.

The aforementioned and other embodiments of the present shall bedescribed in greater depth in the drawings and detailed descriptionprovided below.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features and advantages of the present invention will beappreciated, as they become better understood by reference to thefollowing detailed description when considered in connection with theaccompanying drawings, wherein:

FIG. 1 illustrates the architecture of a distributed inspection networkthat uses non-intrusive X-ray scanning, according to one embodimentdescribed in the present specification;

FIG. 2 is a diagram presenting the overall system architecture of theimaging system of described in the present specification, in oneembodiment;

FIG. 3 illustrates an exemplary interface for a service post, asemployed in one embodiment of the system described in the presentspecification;

FIG. 4 depicts an exemplary interface for presenting manifestinformation, as employed in one embodiment of the system described inthe present specification;

FIG. 5 shows an exemplary user interface screen for a data center, asemployed in one embodiment of the system described in the presentspecification;

FIG. 6 shows another exemplary user interface screen for a data center,as employed in one embodiment of the system described in the presentspecification;

FIG. 7 is flowchart illustrating one process for preparing a featuresdatabase, according to one embodiment of the system described in thepresent specification;

FIG. 8 illustrates the use of the features database described withrespect to FIG. 7 to determine if cargo under inspection matchesmanifest information;

FIG. 9 illustrates the process of using the features database describedwith respect to FIG. 7 to determine if cargo under inspection matchesthe manifest, when there is more than one type of cargo present in theshipment;

FIG. 10 illustrates how currently scanned images may be visuallycompared with images from the database of the present specification todetermine if cargo matches the manifest; and

FIG. 11 illustrates the segregation of cargo into various cargo typesbased upon scanned images.

DETAILED DESCRIPTION

In one embodiment, the present specification discloses a system forautomatically presenting manifest information when a cargo container ora light vehicle is being inspected using non-intrusive X-ray imagingtechniques. This allows the operator or inspector to quickly ascertainand verify the contents of the cargo container or vehicle that iscurrently being inspected.

In one embodiment, manifest data is imported via an applicationintegrated with an X-ray detection system deployed at checkpoints orservice posts. In one embodiment, the application works within theframework of a distributed network, wherein a service post is connectedto a regional center, whereby an operator can analyze the X-ray image ofthe cargo in conjunction with the manifest data. When the X-ray imageand manifest data has been analyzed, the service post which performedthe non-intrusive X-ray scan will be notified automatically by theapplication integrated with the X-ray system. This allows the servicepost operator to make a decision to either release the cargo or to holdthe cargo for further inspection.

The present specification discloses multiple embodiments. The followingdisclosure is provided in order to enable a person having ordinary skillin the art to practice the invention. Language used in thisspecification should not be interpreted as a general disavowal of anyone specific embodiment or used to limit the claims beyond the meaningof the terms used therein. The general principles defined herein may beapplied to other embodiments and applications without departing from thespirit and scope of the invention. Also, the terminology and phraseologyused is for the purpose of describing exemplary embodiments and shouldnot be considered limiting. Thus, the present specification is to beaccorded the widest scope encompassing numerous alternatives,modifications and equivalents consistent with the principles andfeatures disclosed. For purpose of clarity, details relating totechnical material that is known in the technical fields related to theinvention have not been described in detail so as not to unnecessarilyobscure the present invention.

One of ordinary skill in the art would appreciate that the featuresdescribed in the present application can operate on any computingplatform including, but not limited to: a laptop or tablet computer;personal computer; personal data assistant; cell phone; server; embeddedprocessor; DSP chip or specialized imaging device capable of executingprogrammatic instructions or code.

It should further be appreciated that the platform provides thefunctions described in the present application by executing a pluralityof programmatic instructions, which are stored in one or morenon-volatile memories, using one or more processors and presents and/orreceives data through transceivers in data communication with one ormore wired or wireless networks.

It should further be appreciated that each computing platform haswireless and wired receivers and transmitters capable of sending andtransmitting data, at least one processor capable of processingprogrammatic instructions, memory capable of storing programmaticinstructions, and software comprised of a plurality of programmaticinstructions for performing the processes described herein.Additionally, the programmatic code can be compiled (either pre-compiledor compiled “just-in-time”) into a single application executing on asingle computer, or distributed among several different computersoperating locally or remotely to each other.

FIG. 1 illustrates the architecture of a distributed inspection networkthat uses non-intrusive X-ray scanning. The components of systemarchitecture are described as follows:

Service Post and Regional Center

Referring to FIG. 1, service post 101 is the point where a non-intrusiveX-ray scan is performed. In one embodiment, manifest data is importedvia an application integrated within an X-ray inspection system deployedat a checkpoint or service posts. It should be noted herein that oneexemplary scanning and inspection system that may be employed with thesystems and methods of the present invention includes, but is notlimited to the Rapiscan Eagle Mobile inspection system. Any suitablesystem for inspecting cargo, cargo containers, and their contents may beemployed. As such, U.S. patent application Ser. Nos. 12/780,910;13/370,941; 13/548,873; 13/532,862; 13/168,440; 13/175,792; 13/433,270;13/281,622; 13/108,039; 12/675,471; 12/993,831; 12/993,832; 12/993,834;12/997,251; 12/919,482; 12/919,483; 12/919,484; 12/919,485; 12/919,486;12/784,630; 12/784,465; 12/834,890; 13/009,765; 13/032,593; 13/368,178;and 13/368,202, all assigned to the assignee of the present inventionrepresent various systems that may be employed with the presentinvention and are herein incorporated by reference in their entirety. Inaddition, U.S. Pat. Nos. 5,638,420; 6,542,580; 7,876,879; 7,949,101;6,843,599; 7,483,510; 7,769,133; 7,991,113; 6,928,141; 7,517,149;7,817,776; 7,322,745; 7,720,195; 7,995,705; 7,369,643; 7,519,148;7,876,879; 7,876,880; 7,860,213; 7,526,064; 7,783,004; 7,963,695;7,991,113; 8,059,781; 8,135,110, 8,170,177; 8,223,919; and 8,243,876 allassigned to the assignee of the present invention represent variousscreening systems that may be employed with the present invention areherein incorporated by reference in their entirety.

Service post 101 further comprises at least one, and preferably a set,of non-intrusive inspection (NII) servers 111 through which the servicepost interfaces with other components of the system. After scanning, theoperator responsible for controlling or operating service post 101 canverify that the X-ray image produced by the non-intrusive X-ray scan isof sufficient quality to be effectively analyzed. In one embodiment, theimage analysis is performed at the regional center 102. In oneembodiment, if the image is incomplete, or is corrupted, black (fromattenuating cargo) or is unacceptable in any manner, the service postoperator may request a rescan. This can happen in cases where the timebetween the scan and analysis is close and the truck is still available.

The servers 111 at the service post 101 comprise standard non-intrusiveinspection software. When a vehicle is about to be scanned, the softwareat the service post queries a predicative or routing softwareapplication 103 to receive an instruction, routing information, or anyother data to identify a target regional center for analysis. Theregional center 102 comprises servers 121 and inspection monitors 122.As a new X-ray image is generated at the service post 101, it istransmitted onward from service post 101 to at least one server 121located at a regional center 102, pursuant to routing informationreceived from the software application 103, for analysis by aninspection operator located at that regional center and for subsequentstorage. It should be appreciated that, typically, the regional center102 and service posts 111 are geographically remote from each other.

In one embodiment, the image is allocated to a regional center and/or anoperator within that regional center via the predictive or routingsoftware 103, but the work is only allocated after the imagetransmission is complete. In one embodiment, to streamline the datatransmission activity, predictive software 103 allocates an image to aregional center 102 before the image has been completely generated.

In one embodiment, in the event of the operator becoming unavailable,such as due to PC failure, log off, etc., another operator in the localregional center is selected automatically by the predictive software103.

Further, the system will fall back on an alternative regional center inthe event of a transmission error. In one embodiment, images arebuffered until a center comes back on line.

In one embodiment, each X-ray inspection image is associated with a GUID(Globally Unique Identifier), which is a unique ID across all systems.The GUID is used for associating each image with its particular manifestdata. In one embodiment, identifying information, such as license plate,CCTV images etc. are also associated with the GUID at the time ofscanning. In one embodiment, the GUID is a 128-bit number displayed inhexadecimal. This information may be transmitted to the inspectionoperators at the regional center, if required.

When the X-ray image and manifest data have been analyzed, the servicepost 101 which performed the non-intrusive X-ray scan is notifiedautomatically by means of a data transmission from a softwareapplication referred to herein as CertScan 105. The CertScan applicationpresents an interface to the operator at the service post 101, whichshows the operator a rolling status of all non-intrusive X-ray scansperformed at that service post, along with relevant data to allow theservice post to either release the cargo or to hold it for furtherinspection. In one embodiment, the relevant data includes license platenumber, work order number, and results of scan. The CertScan applicationsystem is also responsible for importing the manifest data associatedwith the cargo or vehicle being scanned. In one embodiment, manifestdata can come in one or more of several forms, such as but not limitedto a) a hardcopy of the manifest; b) from a computer owned and connectedto the customer database; or c) from a customer database accesseddirectly by CertScan. The format in which manifest data is supplieddepends on the customer, and their local requirements and regulations.This is described in greater detail below with respect to the Collectionof Manifest Data.

Predictive Software

The predictive software operates to optimally balance the loaddistribution of image analysis among multiple regional centers andoperators. The predictive software processes meta-data from the regionalcenters and service post connectors to analyze and predict the bestdistribution of images to operators. For example, predictive software103 uses historical meta-data on inspection queue lengths, workload,contention time and a randomization factor to varying degrees, toallocate work to regional centers and individual operators.

Logging and Validation

At various stages of the process, the system provides localized andcentralized logging, auditing, and accounting for each X-ray scanningoperator and X-ray image inspection analyst action. Centralized loggingis provided at the data center 104. During all steps of the process,from scanning, through inspection to search, the system provides ajournal of actions for each non-intrusive X-ray scan and X-ray imageinspection analysis.

Inspection Performance and Metrics

In one embodiment, the system records several X-ray image inspectionmetrics, such as image coverage, tools used, mean time to inspect, timepending, among other variables. These metrics can yield information foroperators/image analysts such as what tools were used (for example,zoom, contrast, brightness, and other parameters), how long it took toanalyze the image, and/or what part of the image was analyzed usingtools. This information can then be applied to measure attentiveness anddiligence of operators. For example, this information may be reviewedfor each X-ray image inspection analyst, and is useful in training,review and performance evaluation. In one embodiment, inspection metricsmay be measured quantitatively and be assigned minimum and maximumvalues, against which the operators' performance may be evaluated.

Besides helping to assess the proficiencies of the analysts, data logsalso allow an assessment of inspection volumes at regional centers andthe speed at which analyses are performed.

In one embodiment, the system provides for secondary X-ray imageinspection, for a percentage of images, or if required on targetedusers. That is, if required in certain cases, the X-ray image inspectionprocess is repeated twice to cross-check results. The second X-ray imageinspection can be assigned to either a purely random X-ray imagescanning operator, or to nominated workstations for quality and trainingpurposes, in various embodiments. The final X-ray image inspectionresult would not be sent to the service post until both inspections arecomplete. If either result is “suspicious”, the suspicious result wouldbe recorded, and any disagreement would be flagged.

In one embodiment, training images may be inserted into the workflow topass on suspicious images to operators as part of their standardworkload. The system then carefully segregates the results from theseimages, without the X-ray scanning operator knowing the difference. Thisallows for discrete and impromptu training of operators.

If a suspicious finding is communicated back to the service post, theoperators can choose to manually open and search the suspicious cargo.In one embodiment, the system allows the operators to record detailedcomments about the manual search process, which can provide both usefulinformation about the suspicious cargo and useful feedback to trainers.

CertScan Software Application

Still referring to FIG. 1, the primary goal of the CertScan application105 is to present manifest information clearly for the non-intrusiveX-ray image analysis inspector to quickly ascertain the contents of thecargo container or light vehicle that is currently being inspected. Theapplication 105 runs on an application server 151 and interfaces with amaster database 152. In one embodiment, the manifest information andrelated data that the CertScan application 105 provides may be importedinto the master database 152 through any suitable means, such as EDI(Electronic Data Interchange), web services, or OCR scanning of manifestdocumentation. The manifest information that is provided by thesesources includes, but is not limited to, the following data elements:

Container Number

Arrival Date

Shipping Line

Bill of Lading Number

Port of Origin

Exporter

Consignee

Container Manifest

Besides use in security inspections, additional related data captured inthe CertScan application database 152 may be used for internalstatistical analysis, financial forecasting and operational reporting.In one embodiment, application 105 generates various reports, includingdaily, weekly, and monthly data related to the expected arrival dates ofcargo containers and light vehicles, as well as data regarding actualcargo containers and light vehicles scanned. In one embodiment, captureddata further includes information such as the number of containersscanned at each site, average to analyze a scan, scans withoutsupporting data, number of scans with threats and without threats, etc.In one embodiment, this data is presented in real time on an userinterface, referred to herein as ‘Dashboard.

In one embodiment, the use of the CertScan system is extended to providereporting through online customer portals or electronic data exchange.Additionally, CertScan may also be extended to provide web services forsupporting “cloud” type solutions. In one embodiment, web servicesinclude obtaining manifest data and publishing or transmitting resultsof the scan along with any anomalies noted. These additional featuresare all value-added services for the security scanning system. Thus, thereports provided by the CertScan application may be coupled with x-rayimages (JPG) which are produced by the scanning software, to create acombined reporting package. These reports may be provided to customersfor their own analysis and audit purposes.

FIG. 2 is a diagram presenting the overall system architecture of theCertScan application (shown as 105 in FIG. 1), according to oneembodiment of the present invention. The hardware for running theCertScan application 200 includes an application server 201 and a masterdatabase 202. The CertScan application provides manifest data to theregional center 203, which is used by the operator in conjunction withthe scanned X-ray image to analyze and determine the disposition of thecargo or light vehicles. In one embodiment, each regional center has agraphical user interface (GUI), the CertScan Application Dashboard orCertScan Dashboard, which shows the analyst all the non-intrusive X-rayscans ready for analysis. Using the CertScan Application Dashboard, theimage analyst can select the X-ray Image to be analyzed. At the time ofselection, CertScan Dashboard displays the cargo and light vehiclemanifest data along with its X-ray image. Once adjudication has beendetermined, the image analyst records the result in a databaseassociated with the CertScan Application. The CertScan Dashboard at theservice post 204 which performed the X-ray scan is then updated with theresult. The result allows the service post operator to take appropriateaction of releasing or holding for further inspection the cargo andlight vehicles.

As mentioned earlier, scan images are packaged with metadata and sentfrom service post 204 to a data center 205 and regional center 203. Themetadata is also processed and loaded into CertScan master database 202.In one embodiment, the scan images and metadata are packaged together asa scant transaction file 206, with a ‘.stf’ extension, for easycommunication between the service post, regional center, data center andthe CertScan application database. In one embodiment, metadata includesinformation such as time of scan, the operator ID, and whether a rescanis required. This data helps establish how long it takes to transmitimages and how long it takes to analyze a scan. This information alsoenables work quality monitoring and statistical reporting.

In one embodiment, the CertScan primary application is a web-basedapplication which resides at the data center 205. The CertScan Dashboardin the data center displays all non-intrusive X-ray scan being performedand all regional centers, as well as all status information. The datacenter also serves as the storage location for all X-ray images.

In one embodiment, the CertScan Application is externally integratedwith web services 207, which may be used to generate reports asdescribed earlier. In one embodiment, the CertScan application isintegrated with the inspection software to provide a comprehensivesolution for efficient non-intrusive X-ray inspection.

Collection of Manifest Data

As described above, manifest data can come in one or more of severalforms, such as but not limited to a) a hardcopy of the manifest; b) froma computer owned and connected to the customer database; or c) from acustomer database accessed directly by CertScan. In one embodiment, theCertScan Application accepts cargo and light vehicle manifest data inmultiple formats including, but not limited to:

Electronic Data Interchange

Formatted Data Files (Fixed Width or XML)

Transportation Management System Interfaces

2D Bar Code Reader

Manifest Documentation

Some methods, such as Electronic Data Interchange (EDI) of formatteddata files may be preferred to facilitate faster import of data into theCertScan master database before the cargo arrives. When using EDI toacquire the cargo container and light vehicle data provided by thecustomer, data integration is accomplished by importation of a formattedflat file. However the application is designed to support other dataexchange formats that are widely accepted by Freight Management Systems(FMS) standards, web services, or OCR scanning of manifestdocumentation. One of ordinary skill in the art would appreciate thatthe system may be configured to accept additional or other forms ofmanifest input.

In one embodiment, a lack of manifest information may be used to detecthidden compartments and contraband such as weapons, nuclear materials,among other contraband. More specifically, in one embodiment, incompleteor otherwise inadequate manifest information may be indicative of cargothat requires further inspection.

Thus, in one embodiment, the present specification includes systems andmethods for automatically and rapidly detecting the presence ofhigh-atomic-number (high-Z) materials such as nuclear materials; nuclearweapons; and, shielding materials that may be used to shield radiationemitted by such materials as well as by radiological dispersal devices,which can prevent them from being detected by radiation detectors. Thepresent specification also includes the detection of other types ofhigh-Z materials that may be smuggled in cargo due to their value, suchas gold and platinum bullion, and works of art and antiquitiescontaining high-Z materials.

The present specification therefore advantageously employs a threatdetection algorithm that uses physical properties such as materialdensity, mass absorption coefficient, and dimension to determine whetherhigh-Z materials are present in the cargo.

The threat detection method and algorithm requires a much shorteranalysis time and, thus, allows for higher system throughput compared toa conventional system, which requires an inspector manually reviewingthe image or cargo for objects that are highly attenuating. For example,if multiple objects that are highly attenuating are identified, theinspector would need to make contrast enhancements with each objectusing a computer and input device, such as mouse. Each object has tothen be evaluated for its total attenuation (or transmission) value byusing the computer to select a region of interest within the object andmaking an estimate of the average attenuation (or transmission) value,which reflects the total attenuation (or transmission) along the X-raypath through the cargo. Before the net attenuation (or transmission) ofthe object can be estimated, the attenuation (or transmission) of thesurrounding background material has to be analyzed. Then, to generate anaverage net attenuation (or transmission) of the object, the backgroundmust be subtracted from the total attenuation (or added to thetransmission). Finally, the inspector must examine the shape and size ofthe object, and combine these estimates with the estimated netattenuation (or transmission) to reach a conclusion of whether theobject represents a threat. This procedure would have to be repeated foreach object and, therefore, if performed accurately, would be a verytime-intensive procedure.

The threat detection process described in the present specification, inone embodiment, operates by first receiving, on a computing platform, aradiographic image of an object from an X-ray imaging system whichtypically comprises a radiation source positioned opposite to, or awayfrom, a detector array. At least part of the area bounded by theradiation source and detector array is an inspection region, throughwhich the cargo being inspected passes, or is positioned. In oneembodiment, the screening system acquires the original image, which isthen processed by the methods described herein. The X-ray imaging systemis in electrical communication, either wired or wirelessly, with thecomputing platform. The threat detection algorithm then performs a firstlevel analysis to generate a first “suspicious object” binary map bymeasuring a number of physical attributes. Each area on the initialbinary map is used as a mask to electronically crop out part of theX-ray radiographic image for analysis, including its surroundingbackground attenuation (or transmission) and physical characteristicssuch as attenuation, size, and shape. Then, a decision is made ofwhether that area or portion could represent a high-Z object. Thisdecision process results in a second binary map, which highlights thoseregions that represent potential high-Z threats.

In using the threat detection method and algorithm with the methods ofthe present specification the threat or no-threat decision time rangesfrom typically less than one second for cargo determined not to have anysuspicious objects, to less than approximately 5 seconds for cargohaving a plurality of objects or areas of interest. U.S. patentapplication Ser. No. 12/780,910, entitled “Systems and Methods forAutomated, Rapid Detection of High Atomic Number Materials” is hereinincorporated by reference in its entirety.

Dashboard for Real-Time Updates

As mentioned earlier, data is presented by the CertScan application inreal time through a GUI referred to herein as a “Dashboard”. TheCertScan Dashboard preferably runs on all the three components of thesystem—the service post, the regional centers and the data center. Inone embodiment, the CertScan Dashboard displays a rolling list ofnon-intrusive X-ray scans, with data elements that are appropriate foreach of the three locations.

In one embodiment, the CertScan application controls the flow of allX-ray image manifest data to ensure all three components have thecontent and data necessary to carry out their operations.

Service Post Dashboard

FIG. 3 illustrates an exemplary GUI (Dashboard) for the service postthat is provided by the CertScan Application. This GUI has the goal ofproviding the service post operator with the optimal information toassist in deciding if the cargo being scanned is to be released or heldfor further inspection. Referring to FIG. 3, the data displayed on theService Post Dashboard may include the container ID number 301, scanstart time 302 and scan end time 303, time of start 304 and time ofcompletion 305 of analysis of image and data at the regional center, thestatus (result) 306, as conveyed by the regional center, and comments307, if any from the regional center analyst. In one embodiment, thestatus or result 306 is indicated visually and by means of color coding.Thus, for example, green 306 a may indicate ‘ready to clear’, red 306 bmay indicate the need for manual or visual inspection, blue 306 c mayindicated ‘under analysis’, and yellow 306 d may represent already‘cleared’.

The CertScan Dashboard located at the service post need not display anyinformation about which regional center performed the X-ray imageanalysis or the identity of the image analyst who performed theanalysis.

Regional Center Dashboard

This CertScan Dashboard aims to provide the regional center imageanalyst with the information required to quickly and efficiently analyzethe X-ray image for potential threats or contraband, and enables theanalyst to record the results of the image inspections.

The image analyst uses the CertScan Dashboard to select an X-ray scanready for analysis. The CertScan Dashboard located at the regionalcenter does not display any information about which service postperformed the non-intrusive X-ray scan or the identity of the servicepost operator who performed the X-ray scan.

In one embodiment, CertScan application interface for the image analystis designed to be easy to use, and presents manifest information in amanner such that the analyst requires minimal time to evaluate the cargocontainer and light vehicle manifest data and record scan results.

The CertScan user interface at the regional center is integrated withthe inspection software to retrieve the cargo container and lightvehicle manifest information once the X-ray scan is complete. Anexemplary interface presenting the manifest information to the imageanalysis inspector is shown in FIG. 4. Referring to FIG. 4, theinterface screen provides manifest data such as shipper ID 401,container number 402, expected date of arrival of shipment 403, type(size) of container 404, and names of the exporter 405 and the consignee406. The screen also includes a manifest table 407 which provides datasuch as description of item (contents), harmonized tariff schedule(HTS), item unit, and unit quantity.

The X-ray image analysis inspector can thus verify if information aboutthe cargo container and light vehicle matches with the scanned images.The image analysis inspector can then record the inspection result inthe interface screen, using the color coded result buttons 408. In mostcases the result will be ‘Cleared’, which is represented by a greenbutton in one embodiment. However, there may be instances where certainareas in the X-ray Image cannot be identified clearly or it isidentified that contents that could be harmful. In these cases there aretwo other results which can be recorded—‘Irregularity’ or ‘PossibleThreat’, represented by yellow and red respectively, in one embodiment.In one embodiment, blue color is used to indicate ‘Rescan required’ incase the image is unreadable. This may happen, for example, due to anenvironmental condition which may affect the quality and clarity of theX-ray image. In this case the cargo and vehicle under inspection need tobe scanned again.

Data Center Dashboard

The data center uses the CertScan Dashboard to select an X-ray scan atany point of its lifecycle. The CertScan Dashboard located at the datacenter displays comprehensive information about the service postsperforming the non-intrusive X-ray scan and the regional center whereanalysis of the X-ray image is being performed.

The CertScan application user interface screens for the Data Centerprovides all the functionality of the regional center, plus otherfunctions. FIG. 5 shows an exemplary user interface screen for the datacenter. Referring to FIG. 5, the interface allows the dater centerpersonnel to search for past scan records 501 as well as un-scannedcargo 502 whose manifest data is loaded in the system. The operator mayalso search for specific details of a cargo by container number 503 orby arrival date range 504. The search yields records for the specificcontainer, which include data such as container type 505, shipper name506, vessel name 507, expected arrival date 508, scan date 509 and scanresults 510.

FIG. 6 illustrates another exemplary screen for the data center thatshows completed scans. Referring to FIG. 6, scan records may be filteredby shipper name 601, or other attributes, such as consignee name,exporter name, date of arrival, among other parameters. In oneembodiment, the completed scan records include container number 602,shipper name 603, vessel name 604, voyage number 605, and expectedarrival date 606.

One of ordinary skill in the art would appreciate that all the interfacescreens may be customized to meet the customer's needs, and data may beselected for display accordingly.

System Logging

In one embodiment, the CertScan application performs logging of allactivities throughout the full non-intrusive X-ray scanning operation.The application log provides information and reports such as:

Timings related to the non-intrusive X-ray scan process

CertScan Application performance monitoring

CertScan Application system health

CertScan Application error traps

One of ordinary skill in the art would appreciate that CertScanapplication log data may be used for internal system monitoring as wellas for reporting based on customer needs.

The applications of the present inventions may be extended to securityinspection at ports, borders, aviation checkpoints as well as supplychain security. The system can import manifest data from a port, borderor aviation data management system, as the case may be, and compare theobtained information with image of container. In one embodiment, thesystem of present invention automatically applies detection algorithmsto the image and provides alerts to operator, if there are anymismatches with the manifest. This ‘Operator Assist’ function enablesthe security personnel to identify threats or other contraband moreefficiently, and they can determine if de-vanning or opening thecontainer is required. In one embodiment, multiple operators work in amatrix or networking environment and review the alarms generatedautomatically by the system. The operators then decide to clear orfurther investigate the alarms. The application of the system may beextended to supply chain security, where devices that are capable ofsending messages through cell phones or satellite networks, may beattached to pallets and containers. These devices may be used to sendalarms remotely to a central monitoring station, along with X-ray andvideo images if there is an alarm.

One of ordinary skill in the art would appreciate that although theprocess of an operator inspecting an image to verify that the cargomatches the manifest is much more efficient than manually opening thecontainer, it still requires significant labor. The labor-intensivenature of the problem is even more evident in applications such asinspecting each railcar in a long train with hundreds of railcars andtrying to identify thousands of cargo types. Often, it is difficult toidentify the cargo from the numerous images in such cases.

To address this problem, in another embodiment, the present invention isdirected towards the analysis of images generated by non-intrusive cargoinspection systems with the goal of improving the efficiency of theprocess to verify that cargo matches the manifest.

For the purpose of this specification, cargo manifest is defined as amanifest that lists all cargo codes carried on a specific shipment.Further, cargo codes may be standard, also known as harmonization codes,or may be provided by various local custom agencies and may be differentdepending on the jurisdiction.

In one embodiment, predetermined image features of inspected cargo withan associated cargo code are computed and compared with featuresassociated with the same cargo code saved in a database. The comparisonresults in a probability that the inspected cargo matches the declaredcargo in the manifest. If the probability is greater than apredetermined threshold, the cargo will be declared as matching themanifest. Otherwise, the cargo does not match the manifest. In anotherembodiment, the probability is presented to the operator and theoperator makes the decision. These processes are illustrated by means offlowcharts in FIGS. 7, 8 and 9.

Referring to FIG. 7, the process of preparing a features database isshown. In the first step 701, the system obtains the image of thecontainer. The image is obtained through non-intrusive scanning at anyof the service posts, as described above. It should be understood bythose of ordinary skill in the art that the radiographic images could begenerated by low, medium or high-energy X-rays, gamma rays, neutrons orother type of radiation. The images could also contain atomic-numberinformation generated from any modality of dual-energy or dual-speciesinspection. The images could be generated by one or more views and couldbe three dimensional reconstructed from the views.

After obtaining the image, the system obtains cargo code associated withthe image, as shown in step 702. Cargo codes are obtained from manifestdata, as described above. Thereafter, features of the image arecomputed, in step 703. Computed features and their standard deviationare then saved in the database along with the number of images used tocompute the features, and are associated with that cargo code, as shownin step 704.

The features include, but not limited to, attenuation, texture, atomicnumber, and/or cargo height. For tomographic and multi-view systems,density is also a useful feature. This also would include elementalcomposition or features derived from the composition for neutron-basedinterrogation. It should be understood by those of ordinary skill in theart that other features not listed here could be used to match thecargos.

In the next step 705, the system checks if any entries for that cargocode are already stored in the database. If so, the system combinesfeatures from the containers with same cargo code. This is shown in step706. The combination of the feature values takes into account the numberof images used to compute the feature value and is weighted accordingly.Also, the user is notified of outlier feature values (values that areoutside the three standard deviations or other selected range) foracceptance before the combination takes place. Thereafter the combinedset of features for that particular cargo code is saved in the database,as shown in step 707. Thus, the features saved in the database per cargocode are computed from a combination of feature values from a largenumber of cargo images with same cargo code. The feature values areupdated as additional cargo images are collected. Additional featurescan also be used computed as their usability becomes available.

FIG. 8 illustrates a method for performing cargo-manifest verificationfor an individual cargo container. In the first step 801, an imagecaptured at a service post is associated with one or more cargo codes,depending on the contents of the shipment as defined in manifest data.Then, the features of the image are computed, in step 802. Thereafter,the system obtains features for that cargo code stored in a database,and compares them to the computed features. This is shown in step 803.The system then determines the probability ‘P’ that cargo matchesmanifest, in step 804. Probability ‘P’ is then compared to a thresholdvalue in step 805. If ‘P’ is greater than the threshold value, itimplies that cargo matches manifest information declared, as shown instep 806. If ‘P’ is less than the threshold value, it indicates that thecontents of the cargo are not the same as declared in the manifest, asshown in step 807.

In one embodiment, the threshold value may be determined in accordancewith the user's preferences. For example, if custom office is using thesystem and they want to detect most contraband even at the expense ofhigher false alarm rate, they may be able to set a high threshold value,such as 90%. Conversely, if the custom agency does not want to have ahigh false alarm rate, they can choose to set a low threshold value,such as 60%. Further, the customer may decide that some categories ofgoods are more important, such as those associated with higher duties,than others and place different thresholds for different types of goods.

Further, before flagging cargo, a predetermined minimum set of imagesmay be used to compute the features. The customer may decide that thefeatures database is complete and more images do not need to be used. Inthis case, there is no need to add more images to the database. However,if the database did not use enough images, or the customer wants toimprove the accuracy of detection, an authorized operator can request toadd more images to the database. The operator should have a highconfidence that the cargo matches the manifest, which is generallyachieved with experience with the type of cargo coded in the manifest ora manual inspection and verification of the container contents.

When a shipment contains more than one type of cargo, the image isanalyzed for different types of cargo and segregated. This process isillustrated in FIG. 9. Referring to FIG. 9, the system first associatesthe image of scanned cargo with manifest information in step 901. Theimage is the analyzed to determine if there are multiple cargos, in step902. The system then segregates each cargo type, as shown is step 903.The segregation of cargo types is discussed in greater detail withrespect to FIG. 11. The features for each cargo type are then computedin step 904 and compared in step 905 with the feature values stored inthe database for each cargo type listed in the manifest. A list ofprobabilities for each segregated cargo is then produced. Thus, ‘P_(i)’is the probability that i^(th) cargo matches with the declared manifest.This is shown in step 906.

Each ‘P_(i)’ is then compared to the threshold value, as shown in step907. One of ordinary skill in the art would appreciate that since thereare more than one type of cargos, there may be more than one thresholdvalue for comparison. The system checks if P_(i) is more than thethreshold value for all “i” in step 908. If P_(i) is more that thethreshold value for all “i”, it is determined that the cargo matches themanifest, as shown in step 909. Otherwise, if one or more segregatedcargos do not match features for one of the cargo codes in the manifest,the cargo(s) will be assigned as not matching the manifest and allcargos that do not match the manifest are listed. This is shown in step910. Alternately, the probabilities for each segregated cargo may bedisplayed to the operator for decision.

In one embodiment, an operator can separate the cargo visually and/orwith the help of tools, such as a “rubber band” type of tool. In anotherembodiment, cargo may be automatically segmented and features of thedifferent parts of the cargo may be computed, as shown in FIG. 11.Segmented regions with similar features are assumed to be same cargo.Thus, on the basis of features cargo in image 1100 of FIG. 11 may besegregated into Type 1 1101 and Type 2 1102.

In another embodiment, the operator inspects the image of a containerwith associated manifest. The operator then requests to retrieve fromthe image database a number of images of cargo with same cargo code. Theoperator compares the images visually and/or aided with various imagemanipulation tools to determine whether the cargo matches the manifest.If the manifest lists more than one cargo code, the operator wouldrequest images for each cargo code for comparison.

Another method to assist the operator for determining whether a cargoimage matches the manifest is to retrieve a number of images from theimage database that have the same cargo type. This is shown in FIG. 10,wherein the current image 1001 of the cargo can be visually compared bythe operator with images 1002 of the same cargo type from database.Additional assistance is provided by displaying values of various cargofeatures of the current and previously imaged cargo. In the example,shown, and by way of example only, the current image 1001 is differentfrom the database images 1002. Thus, the operator should make a decisionthat the cargo does not match the manifest, because the current image isdifferent from those in the database.

The above examples are merely illustrative of the many applications ofthe system of present invention. Although only a few embodiments of thepresent invention have been described herein, it should be understoodthat the present invention might be embodied in many other specificforms without departing from the spirit or scope of the invention.Therefore, the present examples and embodiments are to be considered asillustrative and not restrictive, and the invention may be modifiedwithin the scope of the appended claims.

We claim:
 1. A system for scanning cargo, the system comprising: atleast one non-intrusive X-ray inspection system configured to perform anon-intrusive X-ray scan and generate X-ray scan images of the cargo,said at least one non-intrusive X-ray inspection system furthercomprising an application configured to import manifest data associatedwith the cargo being scanned; and at least one server configured toreceive the X-ray scan images and associated manifest data from the atleast one non-intrusive X-ray inspection system, obtain cargo codeinformation from manifest data, retrieve stored images associated withthe cargo code from an image database, determine a probability that theX-ray scan images of the contents of the cargo are of a same type asspecified in the manifest data by comparing features of the scan imagesto features of the stored images, compare the probability against athreshold value, and, based on the comparison, generate an alarm.
 2. Thesystem of claim 1, wherein the threshold value may be manually set basedon an acceptable false alarm rate.
 3. The system of claim 1, whereineach X-ray scan image is associated with a unique identifier beforetransmission to the at least one server.
 4. The system of claim 1wherein the at least one server is located at a regional center remotefrom the at least one non-intrusive X-ray inspection system.
 5. Thesystem of claim 4, further comprising a database located at the regionalcenter, wherein the X-ray scan images, their associated uniqueidentifiers and manifest data, and results of analyses at the regionalcenter are stored in the database.
 6. A method for inspecting cargo,comprising: scanning the cargo at a service post, wherein the servicepost comprises: at least one non-intrusive X-ray inspection systemconfigured to perform a non-intrusive X-ray scan and generate X-ray scanimages, wherein the at least one non-intrusive X-ray inspection systemfurther comprises an application for importing manifest data associatedwith the cargo being scanned; and a server configured to receive X-rayscan images and associated manifest data from the at least onenon-intrusive X-ray inspection system; using the application, importingmanifest data associated with the cargo being scanned; transmitting theX-ray scan images and associated manifest data to a regional centerremote from the service post; at the regional center, obtaining cargocode information from the manifest data; at the regional center,retrieving stored images associated with the cargo code information froman image database; at the regional center, comparing features of theX-ray scan images to features of the stored images to determine aprobability that the contents of the cargo match the manifest data; atthe regional center, comparing the probability against a thresholdvalue; and at the regional center, generating an alarm based on thecomparison of the probability against the threshold value.
 7. The methodof claim 6, wherein the regional center generates an output based on thecomparison and communicates the output to the service post.
 8. Themethod of claim 6, wherein each of the X-ray scan images is associatedwith a unique identifier before transmission from the service post to aregional center.
 9. The method of claim 8, wherein the X-ray scanimages, their associated unique identifiers and manifest data, andoutput are stored in a database.
 10. The method of claim 6 furthercomprising computing features of the contents of the cargo from theX-ray scan images, wherein said features include at least oneattenuation, texture, cargo height, density or atomic number.
 11. Themethod of claim 6, wherein the cargo code information is indicative of atype of cargo.
 12. The method of claim 11 further comprising segregatingthe X-ray scan images according to cargo types if the cargo isassociated with more than one cargo code.
 13. The method of claim 12,wherein each segregated part of the X-ray scan images is compared to thestored images associated with each of the more than one cargo code. 14.The method of claim 13, wherein comparing the X-ray scan images to thestored images is performed automatically.